<html>
<head><meta charset="utf-8"><title>mutex-usage · wg-async-foundations · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/index.html">wg-async-foundations</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/mutex-usage.html">mutex-usage</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="219155960"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/mutex-usage/near/219155960" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Didrik Nordstrom <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/mutex-usage.html#219155960">(Dec 08 2020 at 00:07)</a>:</h4>
<p>Hi all, wanted to check in on the futures mutex and its usage after a discussion we had on Fuchsia. Is the futures mutex intended to support holding the guard across await points? If not, is its purpose solely to unblock the executor in high-contention use-cases? As such, is the futures mutex solely a matter of performance tradeoffs compared to the sync mutex? (Hearsay has it that the sync mutex performs better in many cases)</p>



<a name="219501631"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/mutex-usage/near/219501631" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Lucio Franco <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/mutex-usage.html#219501631">(Dec 10 2020 at 16:58)</a>:</h4>
<p>Yeah, I think you nailed it. Basically, high contention or needing to hold it across await points. The regular mutex will be always faster. I think we go over this in the tokio guide too.</p>



<a name="219508454"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/mutex-usage/near/219508454" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Taylor Cramer <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/mutex-usage.html#219508454">(Dec 10 2020 at 17:48)</a>:</h4>
<p>Yep, the goal of the futures mutex is to be able to hold it across await points. Holding a normal mutex across await points will lead to deadlocks.</p>



<a name="219538694"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/mutex-usage/near/219538694" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Didrik Nordstrom <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/mutex-usage.html#219538694">(Dec 10 2020 at 21:49)</a>:</h4>
<p>Ok, cool. We should document that, an I'm happy to contribute.</p>



<a name="219539004"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/mutex-usage/near/219539004" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Didrik Nordstrom <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/mutex-usage.html#219539004">(Dec 10 2020 at 21:52)</a>:</h4>
<p>I do have a follow-up question, because there was some disagreement on "good practices" during the discussion. Basically, one argument was that async mutex should NOT be used for holding across await points (but only for high-contention cases). One reason for this is that the guard can be dropped implicitly by dropping/cancelling the future that holds it, which can lead to logically inconsistent state.</p>



<a name="219539059"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/mutex-usage/near/219539059" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Didrik Nordstrom <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/mutex-usage.html#219539059">(Dec 10 2020 at 21:53)</a>:</h4>
<p>The sync mutex has a concept of poisoning which protects against "interruptions in the critical section", if you will.</p>



<a name="219539243"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/mutex-usage/near/219539243" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Didrik Nordstrom <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/mutex-usage.html#219539243">(Dec 10 2020 at 21:55)</a>:</h4>
<p>To be clear, I'm talking not so much about the language semantics, which are clear, but about failure modes that can be really nasty to predict/detect/debug.</p>



<a name="219541178"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/mutex-usage/near/219541178" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Didrik Nordstrom <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/mutex-usage.html#219541178">(Dec 10 2020 at 22:10)</a>:</h4>
<p>The way I see the solution space:</p>
<ol>
<li>Preserve the same API. Communicate that dropping futures is "dangerous".</li>
<li>Preserve the same API. Communicate to users that ALL async functions can be terminated at await points, as such they need to make sure that every combination of partial execution of a future results in an acceptable end state.</li>
<li>Preserve the same API. Discourage holding the guard across await points entirely. Optionally add APIs or must_not_await lints to make it harder to hold it by accident. <em>If</em> they do so anyway, they are "on their own", essentially leaving them with the same problem space of (1) and (2).</li>
<li>Provide an alternative API which supports poisoning in async mutices, e.g.<code>async fn with&lt;O&gt;(&amp;self, FnOnce(&amp;mut Self::T) -&gt; Future&lt;Output=O&gt;) -&gt; O</code>. As a result <code>lock()</code> will need to return a <code>Result&lt;Guard, PoisonError&gt;</code> similar to the sync mutex, which is a breaking change.</li>
</ol>



<a name="219541928"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/mutex-usage/near/219541928" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Didrik Nordstrom <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/mutex-usage.html#219541928">(Dec 10 2020 at 22:17)</a>:</h4>
<p>My commentary:</p>
<ol>
<li>Caller responsibility. Makes it easier to implement async code correctly. We get weakest-link issue, where if ONE user drops futures carelessly, it can cause non-local problems. Essentially, nullify the "just drop a future to cancel it" mantra.</li>
<li>Implementor responsibility. Makes it harder to implement async code correctly, you have to reason about early termination at all times.</li>
</ol>
<p>(Note that 1 and 2 applies to all async Rust code, and probably needs an answer anyway)</p>
<ol start="3">
<li>Limits the usefulness of async mutices, but also reduces the amount of unexpected behavior.</li>
<li>Explicit failure mode, which makes mutex more tedious to use, but can catch failures that are otherwise cannot be detected. Breaking change, not great...</li>
</ol>



<a name="219542342"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/mutex-usage/near/219542342" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Lucio Franco <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/mutex-usage.html#219542342">(Dec 10 2020 at 22:20)</a>:</h4>
<p>Wow yeah that is a very interesting point I've not thought about....Ill have to think on what is right here. I would argue that canceling the future is a user thing and therefore its a logical bug but FOR sure something to document. The lint should help here. I think this is just something we need to call out rather than fix the api to make this impossible to happen.</p>



<a name="219543013"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/mutex-usage/near/219543013" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Didrik Nordstrom <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/mutex-usage.html#219543013">(Dec 10 2020 at 22:27)</a>:</h4>
<p>Yes, probably the (1) and (2) choices needs to be answered for async Rust in general, independent of mutices.</p>
<p>I've heard the mantra "just drop a future if you're no longer interested in it". To me, this directly implies (2), that implementors need to <em>think</em> of async functions as "can be terminated at _any_ await-point". My take is that this makes async code very different from sync code, and introduces a huge mental overhead and is likely to be overlooked as a result. I may overestimate the impact of this though..</p>



<a name="219543308"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/mutex-usage/near/219543308" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Didrik Nordstrom <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/mutex-usage.html#219543308">(Dec 10 2020 at 22:30)</a>:</h4>
<p>A non-mutex example of a function that can cause a "logically inconsistent" state:</p>
<div class="codehilite"><pre><span></span><code>async fn foo(&amp;mut self) {
  self.bar = X;
  do_something_async().await; // can terminate here
  self.baz = Y;
}
</code></pre></div>



<a name="219559224"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/mutex-usage/near/219559224" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/mutex-usage.html#219559224">(Dec 11 2020 at 02:28)</a>:</h4>
<p>another suggestion that came up was a non-async <code>with</code> method on the mutex, i.e., at a glance you can tell that the whole critical section will execute</p>



<a name="219559304"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/mutex-usage/near/219559304" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/mutex-usage.html#219559304">(Dec 11 2020 at 02:29)</a>:</h4>
<p>I'm not sure how useful that would be, though.</p>



<a name="219559536"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/mutex-usage/near/219559536" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/mutex-usage.html#219559536">(Dec 11 2020 at 02:33)</a>:</h4>
<p>I don't think "Drop to cancel" is likely to change. That's a part of the core futures contract. Personally haven't seen it cause problems yet, though I can see how the expectations around sync mutexes in particular can incorrectly carry over in a way that could lead to bugs.</p>



<a name="219653933"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/mutex-usage/near/219653933" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthias247 <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/mutex-usage.html#219653933">(Dec 11 2020 at 19:55)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116883">tmandry</span> <a href="#narrow/stream/187312-wg-async-foundations/topic/mutex-usage/near/219559536">said</a>:</p>
<blockquote>
<p>I don't think "Drop to cancel" is likely to change. That's a part of the core futures contract. Personally haven't seen it cause problems yet, though I can see how the expectations around sync mutexes in particular can incorrectly carry over in a way that could lead to bugs.</p>
</blockquote>
<p>I still would like to be able to opt-out of it, since I think it's surprising to users. I don't think anyone from the now 2 digit Rust users in my org is aware about this besides me. And I've seen issues with it - even made them myself :)</p>



<a name="219654167"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/mutex-usage/near/219654167" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthias247 <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/mutex-usage.html#219654167">(Dec 11 2020 at 19:57)</a>:</h4>
<p>I had the proposal for it showcased at <a href="#narrow/stream/187312-wg-async-foundations/topic/run.20to.20completion.20async.20fn">https://rust-lang.zulipchat.com/#narrow/stream/187312-wg-async-foundations/topic/run.20to.20completion.20async.20fn</a></p>
<p>and actually wanted to get back to it soonish, but had to work on some higher priority things in between</p>



<a name="219657142"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/mutex-usage/near/219657142" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dirkjan Ochtman <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/mutex-usage.html#219657142">(Dec 11 2020 at 20:22)</a>:</h4>
<p>See also <a href="https://github.com/rust-lang/rust/pull/79434">https://github.com/rust-lang/rust/pull/79434</a>, which I guess is relevant?</p>



<a name="219657184"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/mutex-usage/near/219657184" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dirkjan Ochtman <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/mutex-usage.html#219657184">(Dec 11 2020 at 20:22)</a>:</h4>
<p>and here's how tokio documents the differences: <a href="https://docs.rs/tokio/0.3.5/tokio/sync/struct.Mutex.html#which-kind-of-mutex-should-you-use">https://docs.rs/tokio/0.3.5/tokio/sync/struct.Mutex.html#which-kind-of-mutex-should-you-use</a></p>



<a name="219839178"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/mutex-usage/near/219839178" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/mutex-usage.html#219839178">(Dec 14 2020 at 13:16)</a>:</h4>
<p>I've run into cases unrelated to mutexes or critical sections where this behavior of futures would have led to data loss if not caught. For instance, an <code>or</code> of two futures where one is buffering/parsing data from a socket or similar. If the other future becomes ready, the data read gets lost.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>